A comparison of question order effects on item-by-item and grid formats: visual layout matters

Abstract Question order effect refers to the phenomenon that previous questions may affect cognitive response process and respondents’ answers. Previous generate a context or frame in which are interpreted. At same time, online surveys, visual design also shift responses. Past empirical research has yielded considerable evidence supporting impact of question on measurement, but few studies have investigated how effects vary with design. Our main was whether different item-by-item formats compared grid formats. The study uses data from an survey experiment conducted non-probability-based panel Hungary, 2019. We used welfare-related 8’th wave ESS. manipulated questionnaire by changing position calls forth negative stereotypes about such social benefits services. further varied presenting separate pages (item-by-item) one grid. results show placing priming right before target item significantly changed attitudes way, significant only when were presented pages. A possible reason behind this finding be respondents engage deeper cognition separately. On other hand, format robust against order, addition, we found little stronger satisficing grids. findings highlight mixing grids surveys introduce measurement inequivalence, especially expected.